Image sensor with flexible pixel summing

ABSTRACT

An image sensor can include pixels that are grouped into subsets of pixels, with each subset including three or more pixels. A method for asymmetrical high dynamic range imaging can include capturing an image of a subject scene using a single integration time for all of the pixels. In a subset of pixels, charge in N pixels is read out and summed together. N represents a number that is between two and one less than a total number of pixels in the subset. Un-summed charge is read out from one pixel in the subset. The un-summed charge and the summed charge are combined when producing a high dynamic range image.

TECHNICAL FIELD

The present invention relates generally to electronic devices, and morespecifically, to image sensors for electronic devices.

BACKGROUND

Cameras and other image recording devices often use one or more imagesensors, such as a charge-coupled device (CCD) image sensor or acomplementary metal-oxide-semiconductor (CMOS) image sensor. When animage of a scene is captured, the scene can include objects that can bepositioned or illuminated in a way that can make it difficult torepresent the objects with acceptable detail. For example, an object inthe scene can be positioned in a shadow, or the object can beilluminated by a bright light source, such as the sun.

The dynamic range of an image sensor quantifies the ability of the imagesensor to adequately image both high light areas in a scene and low darkareas or shadows in the scene. In general, the dynamic range of an imagesensor is less than that of the human eye. The limited dynamic range ofan image sensor can result in an image losing details in the brighterareas or in the darker areas of the scene.

A variety of algorithms have been produced to improve the dynamic rangeof image sensors. One such algorithm varies the integration times (thetime light is collected) of the pixels in the image sensor, whichproduces multiple images of a scene. For example, some pixels can have ashorter integration time while other pixels can have a longerintegration time. The pixels with the shorter integration time canbetter capture the brighter areas in a scene and the pixels with thelonger integration time can better capture darker areas in the scene.The charge or signals output from the pixels having the shorter andlonger integration times can be combined to produce a final high dynamicrange image that has more detail in the lighter and darker areas of theimage.

However, when integration times of the pixels are varied, the final highdynamic range image can include undesirable motion artifacts. Since thefinal high dynamic range image is essentially a combination of twoimages, one image captured with the shorter integration time and anotherimage captured with the longer integration time, objects in the scenecan move in between the times the two images are captured. Thus, thescene represented in the image captured with the shorter integrationtime can differ from the scene represented in the image captured withthe longer integration time. This difference can produce motionartifacts, such as blurring, in the combined final high dynamic rangeimage.

SUMMARY

In one aspect, an image sensor can include pixels that are grouped intosubsets of three or more pixels. Each pixel can include a photodetectorand a transfer transistor. The transfer transistors in a subset areconnected between the photodetectors in the subset and a common node.Thus, the photodetectors in each subset are operably connected to arespective common node. A method for flexible pixel summing includes fora subset of three or more pixels, selecting N pixels to include in asumming operation, where N represents a number that is between two and atotal number of pixels in the subset of pixels. The charge in the Npixels is summed together by transferring the charge from thephotodetectors in the N pixels to the common node. The charge can betransferred sequentially, simultaneously, or in various combinations.The summed charge is then read out from the common node.

In another aspect, a color filter array can be disposed over theplurality of pixels in an image sensor. The color filter array includesfilter elements and a filter element can be disposed over each pixel. Afilter element can restrict the wavelengths of light that are incidentupon the pixel underlying the filter element. A color filter array canbe used to filter light representing one or more colors.

In another aspect, an image sensor includes pixels that are grouped intosubsets of three or more pixels. A method for asymmetrical high dynamicrange imaging can include capturing an image of a subject scene using asingle integration time for all of the pixels and in a subset of pixels,reading out charge in N pixels and summing the charge together. Nrepresents a number that is between two and one less than the totalnumber of pixels in the subset of pixels. Un-summed charge is read outfrom one pixel in the subset. The un-summed charge and the summed chargecan be combined to produce a high dynamic range image. The method can beperformed for each color plane in a color filter array. Alternatively,charge from two or more color planes can be summed and/or combined. Byway of example only, for monochrome high dynamic range imaging, two ormore color planes can be summed.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the invention are better understood with reference to thefollowing drawings. The elements of the drawings are not necessarily toscale relative to each other. Identical reference numerals have beenused, where possible, to designate identical features that are common tothe figures.

FIG. 1A illustrates a front perspective view of an electronic deviceincluding one or more cameras in an embodiment;

FIG. 1B depicts a rear perspective view of the electronic device of FIG.1A;

FIG. 2 illustrates a simplified block diagram of the electronic deviceof FIG. 1 in an embodiment;

FIG. 3 depicts a cross-section view of the electronic device of FIG. 1Ataken along line 3-3 in FIG. 1A;

FIG. 4 illustrates a simplified block diagram of one example of an imagesensor that is suitable for use as image sensor 302 in an embodiment;

FIG. 5 depicts a simplified schematic view of a pixel suitable for usein an image sensor in an embodiment;

FIG. 6 illustrates one example of a color filter array suitable for usewith an image sensor in an embodiment;

FIG. 7 depicts a Bayer color filter array pattern;

FIG. 8 illustrates one example of a shared pixel architecture in anembodiment;

FIG. 9 depicts the Bayer color filter array pattern for the sixteenpixels shown in FIG. 8;

FIG. 10 illustrates another example of a shared pixel architecture in anembodiment;

FIG. 11 is a flowchart of a method for flexible pixel summing in anembodiment; and

FIG. 12 is a flowchart of a method for asymmetrical high dynamic rangeimaging in an embodiment.

DETAILED DESCRIPTION

Embodiments described herein provide an image sensor or image capturedevice that is capable of performing flexible pixel summing and/orasymmetrical HDR imaging. Pixels in the image sensor can be grouped intosubsets of three or more pixels. Each pixel can include a photodetectorand a transfer transistor. The transfer transistors in a subset areconnected between the photodetectors in the subset and a separate commonnode. Thus, the photodetectors in each subset are operably connected toa respective common node. Flexible pixel summing can include, for asubset of three or more pixels, selecting N pixels to include in asumming operation, where N represents a number that is between two and atotal number of pixels in the subset of pixels. The charge in the Npixels is summed together by transferring the charge from thephotodetectors in the N pixels to the common node. The charge can betransferred sequentially, simultaneously, or in various combinations.The summed charge is then read out from the common node.

With flexible pixel summing, charge in any number of pixels within asubset of pixels can be summed together. In some embodiments, chargesumming with two pixels can be performed as vertical two pixel summing,horizontal two pixel summing, diagonal two pixel summing, and four pixelsumming. Alternatively, three or more pixels can be summed together.Charge representing the same color or different colors can be summedtogether.

With asymmetrical high dynamic range imaging, the pixels in an imagesensor can be grouped into subsets of pixels and each subset can includethree or more pixels. An image of a subject scene is captured using asingle integration time for all of the pixels in the image sensor. In asubset of pixels, charge from N pixels can be read out and summedtogether. N represents a number that is between two and one less than atotal number of pixels in the subset of pixels. Un-summed charge can beread out of one pixel in the subset of pixels. The un-summed charge canbe combined with the summed charge when a high dynamic range image isproduced.

Directional terminology, such as “top”, “bottom”, “front”, “back”,“leading”, “trailing”, etc., is used with reference to the orientationof the Figure(s) being described. Because components in variousembodiments can be positioned in a number of different orientations, thedirectional terminology is used for purposes of illustration only and isin no way limiting. When used in conjunction with layers of an imagesensor wafer, image sensor die, or corresponding image sensor, thedirectional terminology is intended to be construed broadly, andtherefore should not be interpreted to preclude the presence of one ormore intervening layers or other intervening image sensor features orelements. Thus, a given layer that is described herein as being formedon, formed over, disposed on, or disposed over another layer may beseparated from the latter layer by one or more additional layers.

Referring now to FIGS. 1A-1B, there are shown front and rear perspectiveviews of an electronic device that includes one or more cameras in anembodiment. The electronic device 100 includes a first camera 102, asecond camera 104, an enclosure 106, a display 110, an input/output(I/O) member 108, and an optional flash 112 or light source for thecamera or cameras. The electronic device 100 can also include one ormore internal components (not shown) typical of a computing orelectronic device, such as, for example, one or more processors, memorycomponents, network interfaces, and so on.

In the illustrated embodiment, the electronic device 100 is implementedas a smart telephone. Other embodiments, however, are not limited tothis construction. Other types of computing or electronic devices caninclude one or more cameras, including, but not limited to, a netbook orlaptop computer, a tablet computer, a digital camera, a printer, ascanner, a video recorder, and a copier.

As shown in FIGS. 1A-1B, the enclosure 106 can form an outer surface orpartial outer surface and protective case for the internal components ofthe electronic device 100, and may at least partially surround thedisplay 110. The enclosure 106 can be formed of one or more componentsoperably connected together, such as a front piece and a back piece.Alternatively, the enclosure 106 can be formed of a single pieceoperably connected to the display 110.

The I/O member 108 can be implemented with any type of input or outputmember. By way of example only, the I/O member 108 can be a switch, abutton, a capacitive sensor, or other input mechanism. The I/O member108 allows a user to interact with the electronic device 100. Forexample, the I/O member 108 may be a button or switch to alter thevolume, return to a home screen, and the like. The electronic device caninclude one or more input members or output members, and each member canhave a single I/O function or multiple I/O functions.

The display 110 can be operably or communicatively connected to theelectronic device 100. The display 110 can be implemented with any typeof suitable display, such as a retina display or an active matrix colorliquid crystal display. The display 110 can provide a visual output forthe electronic device 100 or function to receive user inputs to theelectronic device. For example, the display 110 can be a multi-touchcapacitive sensing touchscreen that can detect one or more user inputs.

The electronic device 100 can also include a number of internalcomponents. FIG. 2 illustrates one example of a simplified block diagramof the electronic device 100. The electronic device can include one ormore processors 200, storage or memory components 202, input/outputinterface 204, power sources 206, and sensors 208, each of which will bediscussed in turn below.

The one or more processors 200 can control some or all of the operationsof the electronic device 100. The processor(s) 200 can communicate,either directly or indirectly, with substantially all of the componentsof the electronic device 100. For example, one or more system buses 210or other communication mechanisms can provide communication between theprocessor(s) 200, the cameras 102, 104, the display 110, the I/O member108, or the sensors 208. The processor(s) 200 can be implemented as anyelectronic device capable of processing, receiving, or transmitting dataor instructions. For example, the one or more processors 200 can be amicroprocessor, a central processing unit (CPU), an application-specificintegrated circuit (ASIC), a digital signal processor (DSP), orcombinations of multiple such devices. As described herein, the term“processor” is meant to encompass a single processor or processing unit,multiple processors, multiple processing units, or other suitablyconfigured computing element or elements.

The memory 202 can store electronic data that can be used by theelectronic device 100. For example, the memory 202 can store electricaldata or content such as, for example, audio files, document files,timing signals, and image data. The memory 202 can be configured as anytype of memory. By way of example only, memory 202 can be implemented asrandom access memory, read-only memory, Flash memory, removable memory,or other types of storage elements, in any combination.

The input/output interface 204 can receive data from a user or one ormore other electronic devices. Additionally, the input/output interface204 can facilitate transmission of data to a user or to other electronicdevices. For example, in embodiments where the electronic device 100 isa smart telephone, the input/output interface 204 can receive data froma network or send and transmit electronic signals via a wireless orwired connection. Examples of wireless and wired connections include,but are not limited to, cellular, WiFi, Bluetooth, and Ethernet. In oneor more embodiments, the input/output interface 204 supports multiplenetwork or communication mechanisms. For example, the input/outputinterface 204 can pair with another device over a Bluetooth network totransfer signals to the other device while simultaneously receivingsignals from a WiFi or other wired or wireless connection.

The power source 206 can be implemented with any device capable ofproviding energy to the electronic device 100. For example, the powersource 206 can be a battery or a connection cable that connects theelectronic device 100 to another power source such as a wall outlet.

The sensors 208 can by implemented with any type of sensors. Examples ofsensors include, but are not limited to, audio sensors (e.g.,microphones), light sensors (e.g., ambient light sensors), gyroscopes,and accelerometers. The sensors 208 can be used to provide data to theprocessor 200, which may be used to enhance or vary functions of theelectronic device.

As described with reference to FIGS. 1A and 1B, the electronic device100 includes one or more cameras 102, 104 and optionally a flash 112 orlight source for the camera or cameras. FIG. 3 is a simplifiedcross-section view of the camera 102 taken along line 3-3 in FIG. 1A.Although FIG. 3 illustrates the first camera 102, those skilled in theart will recognize that the second camera 104 can be substantiallysimilar to the first camera 102. In some embodiments, one camera mayinclude a global shutter configured image sensor and one camera caninclude a rolling shutter configured image sensor. In other examples,one camera can include an image sensor with a higher resolution than theimage sensor in the other camera.

The cameras 102, 104 include an imaging stage 300 that is in opticalcommunication with an image sensor 302. The imaging stage 300 isoperably connected to the enclosure 106 and positioned in front of theimage sensor 302. The imaging stage 300 can include conventionalelements such as a lens, a filter, an iris, and a shutter. The imagingstage 300 directs, focuses or transmits light 304 within its field ofview onto the image sensor 302. The image sensor 302 captures one ormore images of a subject scene by converting the incident light intoelectrical signals.

The image sensor 302 is supported by a support structure 306. Thesupport structure 306 can be a semiconductor-based material including,but not limited to, silicon, silicon-on-insulator (SOI) technology,silicon-on-sapphire (SOS) technology, doped and undoped semiconductors,epitaxial layers formed on a semiconductor substrate, well regions orburied layers formed in a semiconductor substrate, and othersemiconductor structures.

Various elements of imaging stage 300 or image sensor 302 can becontrolled by timing signals or other signals supplied from a processoror memory, such as processor 200 in FIG. 2. Some or all of the elementsin the imaging stage 300 can be integrated into a single component.Additionally, some or all of the elements in the imaging stage 300 canbe integrated with the image sensor 302, and possibly one or moreadditional elements of the electronic device 100, to form a cameramodule. For example, a processor or a memory may be integrated with theimage sensor 302 in some embodiments.

Referring now to FIG. 4, there is shown a top view of one example of animage sensor suitable for use as image sensor 302 in an embodiment. Theimage sensor 400 can include an image processor 402 and an imaging area404. The imaging area 404 can be implemented as a pixel array thatincludes pixels 406. In the illustrated embodiment, the pixel array isconfigured in a row and column arrangement. However, other embodimentsare not limited to this configuration. The pixels in a pixel array canbe arranged in any suitable configuration, such as, for example, ahexagon configuration.

The imaging area 404 may be in communication with a column select 408through one or more column select lines 410 and a row select 412 throughone or more row select lines 414. The row select 412 selectivelyactivates a particular pixel 406 or group of pixels, such as all of thepixels 406 in a certain row. The column select 408 selectively receivesthe data output from the select pixels 406 or groups of pixels (e.g.,all of the pixels with a particular column).

The row select 412 and/or the column select 408 may be in communicationwith the image processor 402. The image processor 402 can process datafrom the pixels 406 and provide that data to the processor 200 and/orother components of the electronic device 100. It should be noted thatin some embodiments, the image processor 402 can be incorporated intothe processor 200 or separate therefrom.

Referring now to FIG. 5, there is shown a simplified schematic view of apixel that is suitable for use as pixels 406 in an embodiment. The pixel500 includes a photodetector (PD) 502, a transfer transistor (TX) 504, asense region 506, a reset (RST) transistor 508, a readout transistor510, and a row select (RS) transistor 512. The sense region 506 isrepresented as a capacitor in the illustrated embodiment because thesense region 506 can temporarily store charge received from thephotodetector 502. As described below, after charge is transferred fromthe photodetector 502, the charge can be stored in the sense region 506until the gate of the row select transistor 512 is pulsed.

One terminal of the transfer transistor 504 is connected to thephotodetector 502 while the other terminal is connected to the senseregion 506. One terminal of the reset transistor 508 and one terminal ofthe readout transistor 510 are connected to a supply voltage (Vdd) 514.The other terminal of the reset transistor 508 is connected to the senseregion 506, while the other terminal of the readout transistor 510 isconnected to a terminal of the row select transistor 512. The otherterminal of the row select transistor 512 is connected to an output line410.

By way of example only, in one embodiment the photodetector 502 isimplemented as a photodiode (PD) or pinned photodiode, the sense region506 as a floating diffusion (FD), and the readout transistor 510 as asource follower transistor (SF). The photodetector 502 can be anelectron-based photodiode or a hole based photodiode. It should be notedthat the term photodetector as used herein is meant to encompasssubstantially any type of photon or light detecting component, such as aphotodiode, pinned photodiode, photogate, or other photon sensitiveregion. Additionally, the term sense region as used herein is meant toencompass substantially any type of charge storing or charge convertingregion.

Those skilled in the art will recognize that the pixel 500 can beimplemented with additional or different components in otherembodiments. For example, a row select transistor can be omitted and apulsed power supply mode used to select the pixel, the sense region canbe shared by multiple photodetectors and transfer transistors, or thereset and readout transistors can be shared by multiple photodetectors,transfer gates, and sense regions.

When an image is to be captured, an integration period for the pixelbegins and the photodetector 502 accumulates photo-generated charge inresponse to incident light. When the integration period ends, theaccumulated charge in the photodetector 502 is transferred to the senseregion 506 by selectively pulsing the gate of the transfer transistor504. Typically, the reset transistor 508 is used to reset the voltage onthe sense region 506 (node 516) to a predetermined level prior to thetransfer of charge from the photodetector 502 to the sense region 506.When charge is to be readout of the pixel, the gate of the row selecttransistor is pulsed through the row select 412 and row select line 414to select the pixel (or row of pixels) for readout. The readouttransistor 510 senses the voltage on the sense region 506 and the rowselect transistor 512 transmits the voltage to the output line 410. Theoutput line 410 is connected to readout circuitry and (optionally animage processor) through the output line 410 and the column select 408.

In some embodiments, an image capture device, such as a camera, may notinclude a shutter over the lens, and so the image sensor may beconstantly exposed to light. In these embodiments, the photodetectorsmay have to be reset or depleted before a desired image is to becaptured. Once the charge from the photodetectors has been depleted, thetransfer gate and the reset gate are turned off, isolating thephotodetectors. The photodetectors can then begin integration andcollecting photo-generated charge.

In general, photodetectors detect light with little or no wavelengthspecificity, making it difficult to identify or separate colors. Whencolor separation is desired, a color filter array can be disposed overthe imaging area to filter the wavelengths of light sensed by thephotodetectors in the imaging area. A color filter array is a mosaic offilter elements with each filter element typically disposed over arespective pixel. A filter element restricts the wavelengths of lightdetected by a photodetector, which permits color information in acaptured image to be separated and identified. FIG. 6 illustrates oneexample of a color filter array suitable for use with an image sensor inan embodiment. The color filter array (CFA) 600 includes filter elements602, 604, 606, 608. Although only a limited number of filter elementsare shown, those skilled in the art will recognize that a CFA caninclude thousands or millions of filter elements.

In one embodiment, each filter element restricts light wavelengths. Inanother embodiment, some of the filter elements filter light wavelengthswhile other filter elements are panchromatic. A panchromatic filterelement can have a wider spectral sensitivity than the spectralsensitivities of the other filter elements in the CFA. For example, apanchromatic filter element can have a high sensitivity across theentire visible spectrum. A panchromatic filter element can beimplemented, for example, as a neutral density filter or a color filter.Panchromatic filter elements can be suitable in low level lightingconditions, where the low level lighting conditions can be the result oflow scene lighting, short exposure time, small aperture, or othersituations where light is restricted from reaching the image sensor.

Color filter arrays can be configured in a number of different mosaics.The color filter array 600 can be implemented as a red (R), green (G),and blue (B) color filter array or a cyan (C), magenta (M), yellow (Y)color filter array. The Bayer pattern is a well know color filter arraypattern. The Bayer color filter array filters light in the red (R),green (G), and blue (B) wavelengths ranges (see FIG. 7). The Bayer colorfilter pattern includes two green color filter elements (Gr and Gb), onered color filter element, and one blue color filter element. The groupof four filter elements is tiled or repeated over the pixels in animaging area to form the color filter array.

Referring now to FIG. 8, there is shown one example of a shared pixelarchitecture in an embodiment. In the illustrated embodiment, sixteenpixels are connected to a shared common node 800. Each pixel 801includes a photodetector 802 and a transfer transistor 804 connectedbetween the photodetector 802 and the common node 800. Readout circuitry806 can be connected to the common node 800. Since the readout circuitry806 is connected to the common node 800, the sixteen pixels share thereadout circuitry 806. By way of example only, the readout circuitry 806can include a sense region, a reset transistor, and a readout transistorthat can be configured as shown in FIG. 5. The sense region, the resettransistor and the readout transistor can be connected to the commonnode 800. A row select transistor can be connected to the readouttransistor.

The gates of each transfer transistor 804 can be selectively pulsed inone embodiment, allowing charge from one or more photodetectors 802 totransfer to the common node 800. Since the transfer transistors 804 caneach be selectively pulsed, the charge from one pixel or multiple pixelscan be transferred separately, in combinations, or simultaneously to thecommon node 800. Thus, charge summing between the sixteen pixels can beflexible in that any combination of pixels can be summed. Charge summingcan be performed with as few as two pixels up to all sixteen pixels. Forexample, the charge from pixels 808 and 814 can be summed together byseparately or simultaneously pulsing the gates of the respectivetransfer transistors, thereby transferring the charge to the common node800. The summed charge can then be readout using some or all of thecomponents in the readout circuitry 806.

Charge summing can occur in the same color plane or in multiple colorplanes. FIG. 9 depicts the Bayer color filter pattern for the sixteenpixels shown in FIG. 8. Within the sixteen pixels, there are fourdifferent color planes. Four pixels are associated with the color red(R), four with the color green (Gr), four with the color green (Gb), andfour with the color blue (B). In FIG. 8, pixels 808, 810, 812, 814correspond to the red filter elements and pixels 816, 818, 820, 822 tothe blue filter elements. In the illustrated embodiment, multipledifferent charge summing options can be implemented with flexible pixelsumming in the same color plane. For example, in the red color plane,charge in pixels 808 and 810 can be summed together and read out, andcharge in pixels 812 and 814 can be summed together and read out.Alternatively, charge in pixels 808 and 812 can be summed together andread out, and charge in pixels 810 and 814 can be summed together andread out. Likewise, charge in pixels 808 and 814 can be summed togetherand read out, and charge in pixels 810 and 812 can be summed togetherand read out. Alternatively, charge in three pixels (e.g., 808, 810,812; 808, 810, 814; 808, 812, 814; or 810, 812, 814) can be summedtogether. And finally, charge in all four pixels 808, 810, 812, 814 canbe summed together and read out.

In some embodiments, charge summing can occur across different colorplanes. For example, charge in one or more red pixels can be summed withcharge in one or more green (Gr and/or Gb) pixels. Alternatively, chargein one or more blue pixels can be summed with charge in one or moregreen (Gr and/or Gb) pixels or with one or more red pixels. Likewise,charge from one or more red pixels, green pixels (Gr and/or Gb), andblue pixels can be summed together.

Thus, charge in any number of pixels that share a common node can besummed together. Thus, charge summing with two pixels can be performedas vertical two pixel summing, horizontal two pixel summing, diagonaltwo pixel summing, and four pixel summing. Alternatively, charge inthree pixels up to sixteen pixels can be summed together. A processingdevice, such as, for example, processor 200 in FIG. 2 or image processor402 in FIG. 4 can be used to select which pixels are summed together.

Flexible pixel summing can also be used with asymmetrical high dynamicrange (HDR) imaging. For illustrative purposes only, asymmetrical HDRimaging will be described in conjunction with a four pixel sharedarchitecture shown in FIG. 10. Four pixels 1000 are connected to ashared common node 1002. Readout circuitry 1004 is connected to thecommon node 1002, allowing the four pixels 1000 to share the readoutcircuitry. The readout circuitry can be configured with any suitablereadout circuitry, such as with the readout circuitry described inconjunction with FIG. 8.

Each pixel 1000 can include a photodetector 1006, 1008, 1010, 1012 and atransfer transistor 1014, 1016, 1018, 1020 connected between thephotodetector and the shared common node 1002. The gates of eachtransfer transistor 1014, 1016, 1018, 1020 can be selectively pulsed inone embodiment, allowing charge from one or more photodetectors totransfer to the common node 1002. Since the transfer transistors 1014,1016, 1018, 1020 can each be selectively pulsed, the charge from one,two, three, or four photodetectors can be transferred separately, incombinations, or simultaneously, and the charge in two or morephotodetectors can be summed together.

Unlike conventional HDR imaging techniques that utilize differentintegration times for the pixels in an imaging array, with asymmetricalHDR imaging, all of the pixels can have the same integration time. A HDRimage can be produced by reading out un-summed charge from one pixelseparately and then summing charge together from two or more pixels andreading out the summed charge. The un-summed charge and the summedcharge can then be combined to produce a final HDR image.

For example, in the embodiment shown in FIG. 10, the gate of a transfertransistor 1014 connected to the photodetector 1006 can be pulsed totransfer the accumulated charge in the photodetector 1006 to the commonnode 1002. The charge can then be read out using some or all of thecomponents in the readout circuitry 1004. The charge read out of the onephotodetector 1006 is un-summed charge in that the charge has not beensummed together with charge from another photodetector. The un-summedcharge from the one photodetector can represent a first image of thescene.

Thereafter, the charge in three pixels can be summed together byselectively transferring the accumulated charge in the threephotodetectors 1008, 1010, 1012 to the common node 1002, and thenreading out the charge using some or all of the components in thereadout circuitry 1004. The accumulated charge in the threephotodetectors can be transferred sequentially, simultaneously, or invarious combinations since the transfer gates of the respective transfertransistors 1016, 1018, 1020 can be selectively pulsed. The summedcharge from the three photodetectors 1008, 1010, 1012 can represent asecond image of the scene. A final HDR image can be obtained bycombining or stitching the first and second images together.

The summed charge from the three photodetectors 1008, 1010, 1012 canhave up to three times the sensitivity as the un-summed charge from theone photodetector 1006. Because all four of the photodetectors 1006,1008, 1010, 1012 had the same integration time when the first and secondimages were captured, the final HDR image can be free of motionartifacts.

Other embodiments are not limited to summing charge from threephotodetectors. Charge in two or more photodetectors can be summedtogether. The summed charge can represent a first image. The first imagecan be combined with un-summed charge to produce a final HDR image.

With the embodiment shown in FIG. 8, asymmetrical HDR imaging can beperformed by reading out the four color planes separately. For example,charge in three pixels associated with the color red can be summedtogether and read out, followed by reading out the charge in the oneremaining pixel associated with the color red. The same procedure isperformed for the pixels associated with the green (Gr), blue, and green(Gb) color planes. The summed charge and the un-summed charge can thenbe combined to produce a final HDR image. For example, summed charge andun-summed charge can be combined first by color plane (e.g., summed andun-summed red combined for red color plane HDR image) and then all colorplanes can be combined to produce the HDR image. Alternatively, all ofthe summed and un-summed charge can be combined at once to produce theHDR image.

In some embodiments, color HDR imaging sums charge by color plane topreserve the color information. Thus, one example sums chargerepresenting the color red together, charge representing the color bluetogether, charge representing the color green (Gr) together, and chargerepresenting the color green (Gb) together. Other embodiments can sumcharge in two or more color planes together to produce a monochrome HDRimage.

Embodiments can construct an image sensor on a singlesemiconductor-based wafer or on multiple semiconductor-based wafers.When a single wafer is used, the components in each pixel reside in oron the single wafer. When multiple wafers are used, the components ineach pixel can be divided between two or more wafers. For example, inthe embodiment illustrated in FIG. 5, the photodetectors and thetransfer transistors can reside on one wafer and the sense regions,reset transistors, readout transistors and row select transistors on adifferent wafer. Alternatively, with the embodiments shown in FIGS. 8and 10, the photodetectors and the transfer transistors can reside on afirst wafer and a common sense region on a second wafer. The reset,readout, and row select transistors can also be formed in or on thesecond wafer and can be shared by two or more photodetectors on thefirst wafer. An interconnect layer is typically used to electricallyconnect the transfer transistors to the sense region or regions.

Referring now to FIG. 11, there is shown a flowchart of a method forflexible pixel summing in an embodiment. Initially, the pixels to beincluded in a summing operation are selected (block 1100). In oneembodiment, the pixels can be selected in real-time prior to, or afteran image is captured. The selection can be based on, for example, thelighting conditions in a scene, the filter elements in a color filterarray, the scene dynamic range, and a desired or given image resolution(summing charge reduces resolution).

Pixel selection allows the pixel summing to be flexible and dynamic inthat the number of pixels connected to a common node that are includedin a summing operation can vary for different summing operations. One ormore logic circuits and/or a processor, such as the processor 200 inFIG. 2 or the image processor 402 in FIG. 4 can be used to select thepixels to be included in the summing operation.

The gates of the respective transfer transistors in the selected pixelsare then pulsed at block 1102 to transfer the accumulated charge fromthe photodetectors connected to the respective transfer transistors to acommon node (block 1104). The summed charge on the common node is thenread out (block 1106). A determination is made at block 1108 as towhether another summing operation is to be performed. If so, the processreturns to block 1100 and repeats until all of summing operations havebeen performed.

Other embodiments can perform the method shown in FIG. 11 differently.Additional blocks can be included or blocks can be omitted. For example,bock 1100 can be omitted in those embodiments where the pixels to beincluded are pre-determined (i.e., fixed) and known.

FIG. 12 is a flowchart of a method for asymmetrical high dynamic rangeimaging in an embodiment. Initially, an image of a subject scene iscaptured using a single integration time for all of the pixels in animaging area (block 1200). Next, as shown in block 1202, the pixels tobe included in a summing operation can be selected. Two or more pixelscan be included in a summing operation. The two or more pixels areconnected to a shared common node in one embodiment. Other embodimentsare not limited to this construction and charge in selected pixels canbe summed after the charge is read out of the imaging area or out of theimage sensor.

The pixels to be included in a summing operation can be selected inreal-time prior to capturing the image or after the image is captured.The determination can be based on, for example, the lighting conditionsin a scene, the color filter elements in a color filter array, or adesired or given image resolution. Pixel selection allows the pixelsumming to be flexible and dynamic in that the number of pixelsconnected to a common node that are included in a summing operation canvary for different summing operations. One or more logic circuits and/ora processor, such as the processor 200 in FIG. 2 or the image processor402 in FIG. 4 can be used to select the pixels to be included in thesumming operation.

In some embodiments, all of the pixels can be read out and the chargeused to produce a HDR image. Other embodiments can read out and discardcharge from some of the pixels and use the charge from the remainingpixels to produce the HDR image. By way of example only, with four redpixels, the charge in two pixels can be summed together (summed charge),the charge from one pixel read out (un-summed charge), and the charge inone pixel read out and discarded. The summed charge and the un-summedcharge can then be combined for a HDR image. Thus, charge in only threeof the four pixels is used to produce the HDR image.

The charge in the selected pixels is then summed together. For example,the gates of the transfer transistors connected to the photodetectors inthe pixels to be included in the summing operation can be pulsed totransfer the accumulated charge from the photodetectors to a common node(block 1204). The charge can be transferred to the common nodesimultaneously, sequentially, or in various combinations. Transferringthe charge to the common node sums the charge together. In oneembodiment, the charge transferred and summed on the common node ischarge that represents, or is associated with, one color plane. Thesummed charge represents a first image of the subject scene.

The summed charge is read out, and then the charge in one pixel is readout. For example, the charge can be read from the common node (block1206) and the gate of a transfer transistor connected to onephotodetector can be pulsed to transfer the accumulated charge from thephotodetector to the common node (block 1208). The un-summed charge isthen read out from the common node at block 1210. The un-summed chargerepresents a second image of the subject scene. A HDR image of thesubject scene can then be created by combining the first and secondimages of the subject scene (block 1212).

The method shown in FIG. 12 is performed for each color plane in oneembodiment. For example, with the Bayer color filter pattern, theprocess is performed for the red (R) color plane, the green Gr colorplane, the green Gb color plane, and for the blue (B) color plane.

Other embodiments can perform the method shown in FIG. 12 differently.For example, bocks 1208 and 1210 can be performed before blocks 1202,1204, and 1206. As another example, one or more blocks can be omitted,such as, for example, block 1202.

The embodiments described herein can provide an image sensor or imagecapture device that is capable of performing flexible pixel summing andasymmetrical HDR imaging. With flexible pixel summing, charge in anynumber of pixels can be summed together. In some embodiments, chargesumming with two pixels can be performed as vertical two pixel summing,horizontal two pixel summing, diagonal two pixel summing, and four pixelsumming. Alternatively, three or more pixels can be summed together.Charge representing the same color or different colors can be summedtogether.

With asymmetrical HDR imaging, all of the pixels in an imaging area canhave the same integration time. Charge from multiple pixels can besummed together and read out to represent a first image of a subjectscene. Un-summed charge from one pixel can be read out separately torepresent a second image of the subject scene. The first and secondimages can then be combined to produce a final HDR image of the subjectscene.

Various embodiments have been described in detail with particularreference to certain features thereof, but it will be understood thatvariations and modifications can be effected within the spirit and scopeof the disclosure. And even though specific embodiments have beendescribed herein, it should be noted that the application is not limitedto these embodiments. In particular, any features described with respectto one embodiment may also be used in other embodiments, wherecompatible. Likewise, the features of the different embodiments may beexchanged, where compatible.

The invention claimed is:
 1. A method for performing flexible pixelsumming in an image sensor that includes a plurality of pixels groupedinto subsets of pixels with each subset of pixels including three ormore pixels operably connected to a separate common node, the methodcomprising: in a subset of pixels, selecting, in real time and based onone or more criterion, N pixels to include in a summing operation, whereN represents a number that is between two and a total number of pixelsin the subset of pixels; summing charge in the N pixels together bytransferring the charge from the N pixels to the common node; andreading out the summed charge from the common node.
 2. The method as inclaim 1, wherein the summed charge on the common node represents onecolor.
 3. The method as in claim 1, wherein the summed charge on thecommon node represents two or more colors.
 4. A method for asymmetricalhigh dynamic range imaging using an image sensor that includes aplurality of pixels, wherein the pixels are grouped into subsets ofpixels and each subset includes three or more pixels, the methodcomprising: capturing an image of a subject scene using a singleintegration time for the plurality of pixels; and in a subset of pixels,reading out charge from N pixels selected from the subset and summingthe charge together, wherein selection is based, at least in part, onone or more criterion and wherein N represents a number that is betweentwo and one less than a total number of pixels in the subset of pixels;reading out un-summed charge from one pixel in the subset of pixels; andcombining the un-summed charge with the summed charge when producing ahigh dynamic range image.
 5. The method as in claim 4, wherein thesummed charge is associated with one color.
 6. The method as in claim 4,wherein the summed charge is associated with two or more colors.
 7. Themethod as in claim 4, further comprising selecting which N pixels in thesubset of pixels to sum prior to summing the charge in the N pixelstogether.
 8. The method as in claim 7, wherein the selected N pixels areassociated with one color.
 9. A method for asymmetrical high dynamicrange imaging using an image sensor that includes a plurality of pixelsgrouped into subsets of pixels and each subset includes three or morepixels operably connected to a separate common node, the methodcomprising: capturing an image of a subject scene using a singleintegration time for the plurality of pixels; and in a subset of pixels,summing charge in N pixels together by transferring the charge from theN pixels to the common node, wherein the N pixels are selected in realtime based, at least in part, on one or more criterion and wherein Nrepresents a number that is between two and one less than a total numberof pixels in the subset of pixels; reading out the summed charge fromthe common node; reading out un-summed charge from one pixel in thesubset of pixels; and combining the un-summed charge with the summedcharge when producing a high dynamic range image.
 10. The method as inclaim 9, wherein the summed charge on the common node is associated withone color.
 11. The method as in claim 9, wherein the summed charge onthe common node is associated with two or more colors.
 12. The method asin claim 9, further comprising selecting which N pixels in the subset ofpixels to sum prior to summing the charge in the N pixels together. 13.An image sensor, comprising: an imaging area that includes a pluralityof pixels, wherein the pixels are grouped into subsets of pixels witheach subset including three or more pixels operably connected to aseparate common node; and a processor for selecting N pixels to includein a summing operation, the pixels being selected in real time andbased, at least in part, on one or more criterion.
 14. The image sensoras in claim 13, further comprising readout circuitry operably connectedto each common node.
 15. The image sensor as in claim 13, wherein eachreadout circuitry operably connected to each common node includes asense region operably connected to the common node and a readouttransistor having a gate connected to the sense region.
 16. The imagesensor as in claim 13, wherein where N represents a number that isbetween two and a total number of pixels in a subset of pixels.
 17. Theimage sensor as in claim 13, wherein where N represents a number that isbetween two and one less than a total number of pixels in a subset ofpixels.
 18. The image sensor as in claim 13, wherein each pixel includesa photodetector and a transfer transistor, wherein the transfertransistor is connected between the photodetector and a respectivecommon node.
 19. The image sensor as in claim 18, wherein a sense regionis connected to each common node.
 20. The image sensor as in claim 13,wherein the one or more criterion comprises one or more of: lightingconditions; filter elements in a color filter array; and imageresolution.